The present inventive concepts relate to data storage, and more particularly, to a storage apparatus and method for autonomous space compaction of data.
It is expected that within the next few years, billions of sensors will be deployed around the world and connected to the Internet Of Things (IOT). The amount of data collected by such sensors will be stored at least temporarily, and in some cases, permanently. The IOT will therefore rely on vast storage databases and underlying storage devices. Storage space compaction is an important aspect of modern data storage. For example, NoSQL database systems periodically merge database files and/or tables to reduce search footprints and maximize free space. Log-structured file systems (e.g., append-only file systems) sometimes implement segment cleaning to improve contiguous space availability for sequential writes. Other conventional approaches include disk defragmentation processes, which clean up invalid space for better performance.
Conventional approaches commonly cause intensive read and/or write activity between host CPUs and storage devices for data compaction. For example, Sorted Strings Tables (SSTables) can be compacted in Apache Cassandra™, an open source distributed database management system, but the intensive communication activity between the host CPUs and storage devices can be a limiting factor for performance. By way of another example, Append Only File (AOF) file rewrites in Redis, an open source key-value cache and store, can be challenging to scale due to the communication overhead. Embodiments of the present inventive concept address these and other limitations in the prior art.